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Abstract —This letter proposes a novel distributed compressed 
estimation scheme for sparse signals and systems based on 
compressive sensing techniques. The proposed scheme consists of 
compression and decompression modules inspired by compressive 
sensing to perform distributed compressed estimation. A design 
procedure is also presented and an algorithm is developed to 
optimize measurement matrices, which can further improve the 
performance of the proposed distributed compressed estimation 
scheme. Simulations for a wireless sensor network illustrate the 
advantages of the proposed scheme and algorithm in terms of 
convergence rate and mean square error performance. 

Index Terms —Distributed compressed estimation, compressive 
sensing, measurement matrix optimization, sensor networks. 


I. Introduction 

D istributed signal processing algorithms are of great 
importance for statistical inference in wireless networks 
and applications such as wireless sensor networks (WSNs) HI, 
ja, a, la. Distributed processing techniques deal with the 
extraction of information from data collected at nodes that are 
distributed over a geographic area III . In this context, for each 
node a set of neighbor nodes collect and process their local 
information, and transmit their estimates to a specific node. 
Then, each specific node combines the collected information 
together with its local estimate to generate improved estimates. 

In many scenarios, the unknown parameter vector to be 
estimated can be sparse and contain only a few nonzero 
coefficients. Many algorithms have been developed in the 
literature for sparse signal estimation fS), lb), El, 10, ID, ifTOl . 
ED, ED, ns, m, CD, CSl, CD, mi- However, these 

techniques are designed to take into account the full dimension 
of the observed data, which increases the computational cost, 
slows down the convergence rate and degrades mean square 
error (MSE) performance. 

Compressive sensing (CS) ED, EQl has recently received 
considerable attention and been successfully applied to diverse 
fields, e.g., image processing ED, wireless communications 
E 2 I and MIMO radar ED- The theory of CS states that an 
S'-sparse signal ojq of length M can be recovered exactly 
with high probability from 0(5 log M) measurements. Math¬ 
ematically, the vector (Dq with dimension D x 1 that carries 
sufficient information about oiq {D <C M) can be obtained 
via a linear model EOl 

LUo = (1) 

where $ € j^dxm measurement matrix. 
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The application of CS to WSNs has been recently investi¬ 
gated in ED, I2D, ED, USD- a compressive wireless sensing 
scheme was developed in Il22l to save energy and bandwidth, 
where CS is only employed in the transmit layer. In ED, 
a greedy algorithm called precognition matching pursuit was 
developed for CS and used at sensors and the fusion center to 
achieve fast reconstruction. However, the sensors are assumed 
to capture the target signal perfectly with only measurement 
noise. The work of ESl introduced a theory for distributed 
CS based on jointly sparse signal recovery. However, in 
ED CS techniques are only applied to the transmit layer, 
whereas distributed CS in the estimation layer has not been 
widely investigated. A sparse model that allows the use of 
CS for the online recovery of large data sets in WSNs was 
proposed in E6\ . but it assumes that the sensor measurements 
could be gathered directly, without an estimation procedure. 
In summary, prior work has focused on signal reconstruction 
algorithms in a distributed manner but has not considered both 
compressed transmit strategies and estimation techniques. 

In this work, we focus on the design of an approach that 
exploits lower dimensions, reduces the required bandwidth, 
and improves the convergence rate and the MSE performance. 
Inspired by CS, we introduce a scheme that incorporates 
compression and decompression modules into the distributed 
estimation procedure. In the compression module, we com¬ 
press the unknown parameter tUp into a lower dimension. As a 
result, the estimation procedure is performed in a compressed 
dimension. After the estimation procedure is completed, the 
decompression module recovers the compressed estimator into 
its original dimension using an orthogonal matching pursuit 
(OMP) algorithm ED, ED, ED- We also present a design 
procedure and develop an algorithm to optimize the measure¬ 
ment matrices, which can further improve the performance 
of the proposed scheme. Specifically, we derive an adap¬ 
tive stochastic gradient recursion to update the measurement 
matrix. Simulation results illustrate the performance of the 
proposed scheme and algorithm against existing techniques. 

This paper is organized as follows. Section II describes 
the system model. In Section III, the proposed distributed 
compressed estimation scheme is introduced. The proposed 
measurement matrix optimization is illustrated in Section IV. 
Simulation results are provided in Section V. Einally, we 
conclude the paper in Section VI. 

Notation: We use boldface uppercase letters to denote 
matrices and boldface lowercase letters to denote vectors. We 
use to denote the inverse operator, {■)^ for conjugate 

transposition and (•)* for complex conjugate. 

II. System Model and Problem Statement 

A wireless sensor network (WSN) with N nodes, which have 
limited processing capabilities, is considered with a partially 
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connected topology. A diffusion protocol is employed although 
other strategies, such as incremental and consensus ED 
could also be used. A partially connected network means that 
nodes can exchange information only with their neighbors as 
determined by the connectivity topology. In contrast, a fully 
connected network means that, data broadcast by a node can be 
captured by all other nodes in the network . At every time 
instant i, the sensor at each node k takes a scalar measurement 
dk{i) according to 

dk{i) = u}QXk{i)+ nk{i), i = l,2,...,l, (2) 

where Xk{i) is the M x 1 input signal vector with zero 
mean and variance cr^ j,, nfc(i) is the noise at each node 
with zero mean and variance j,. From (|2|l, we can see that 
the measurements for all nodes are related to an unknown 
parameter vector cuo with size M x 1 that should be estimated 
by the network. We assume that ujq is a sparse vector with 
S M non-zero coefficients. The aim of such a network is 
to compute an estimate of Wg in a distributed fashion, which 
minimizes the cost function 


N 




(3) 




where E{-} denotes expectation. Distributed estimation of 
ljq is appealing because it provides robustness against noisy 
measurements and improved performance as reported in ID, 
EQI, ED- To solve this problem, a cost-effective technique is 
the adapt-then-combine (ATC) diffusion strategy III 

= uJk{i) + fJ-kXk{i) [dk{i) - {i)xk{i)]*, 


iVk{i + l)= E Ckitpiii), 
l&ATk 

where A4 indicates the set of neighbors for node k, is 

the local estimator of node fc, |A/fc | denotes the cardinality of 
A4 and Cki is the combination coefficient, which is calculated 
with respect to the Metropolis rule 


max(|A4|,|Afi|) > 

Ckl = 0, 

Cfcfc = 1 — E 

l^^^k/k 


and should satisfy 


= \,l eMk^k. 


(5) 


(6) 


Existing distributed sparsity-aware estimation strategies, e.g., 
0 , 0 , 13 , are designed using the full dimension signal space, 
which reduces the convergence rate and degrades the MSE 
performance. In order to improve performance, reduce the re¬ 
quired bandwidth and optimize the distributed processing, we 
incorporate at each node of the WSN the proposed distributed 
compressed estimation scheme based on CS techniques, to¬ 
gether with a measurement matrix optimization algorithm. 

III. Proposed Distributed Compressed Estimation 
Scheme 

In this section, we detail the proposed distributed com¬ 
pressed estimation (DCE) scheme based on CS. The proposed 



After the final iteration 1 

Fig. 1. Proposed Compressive Sensing Modules 

scheme, depicted in Eig. [D employs compression and de¬ 
compression modules inspired by CS techniques to perform 
distributed compressed estimation. In the proposed scheme, at 
each node, the sensor first observes the M x 1 vector Xk{i), 
then with the help of the D x M measurement matrix obtains 
the compressed version Xk{i), and performs the estimation of 
ujq in the compressed domain. In other words, the proposed 
scheme estimates the D x 1 vector (Dq instead of the M x 1 
vector ujq, where D M and the D-dimensional quantities 
are designated with an overbar. At each node, a decompression 
module employs a D x M measurement matrix and a 
reconstruction algorithm to compute an estimate of cug- One 
advantage for the DCE scheme is that fewer parameters need 
to be transmitted between neighbour nodes. 
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Fig. 2. Proposed DCE Scheme 

We start the description of the proposed DCE scheme with 
the scalar measurement dk{i) given by 

dk{i) = i^oXk{i)+nk{i), t = l,2,...,I, (7) 

where d>o = ^k<^o and ifc(z) is the 0x1 input signal vector. 
This operation is depicted in Eig. [Das the compression module. 

Eig. [2] illustrates the proposed DCE scheme. The scheme 
can be divided into three steps: 

• Adaptation 

In the adaptation step, at each time instant 2=1,2, ..., I, each 
node fc=I,2, ..., N, generates a local compressed estimator 
'0fc(*) through 

ipkii) = d>fc(i) -f ^ik{i)el{i)xk{i), (8) 

where efe(i) = dfc(i) - u>^{i)xk{i) and 

• Information exchange 

Given the network topology structure, only the local com¬ 
pressed estimator will be transmitted between node k 
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and all its neighbor nodes. The measurement matrix will 
be kept locally. 

• Combination 

At each time instant i=l,2, . . . , I, the combination step starts 
after the information exchange is finished. Each node will 
combine the local compressed estimators from its neighbor 
nodes and itself through 

u)kii + l) = ^ Cki'ipiii), (9) 

l&Afk 

to compute the updated compressed estimator u)k{i + 1). 

After the final iteration I, each node will employ the OMP 
reconstruction strategy to generate the decompressed estimator 

Other reconstruction algorithms can also be used. 
The decompression module described in Fig. [T] illustrates the 
details. In summary, during the DCE procedure, only the 
local compressed estimator will be transmitted over the 

network resulting in a reduction of the number of parameters 
to be transmitted from M to D. The proposed DCE scheme 
is given in Table U 

The computational complexity of the proposed DCE scheme 
is 0{NDI + ND^), where N is the number of nodes in the 
WSN and I is the number of time instants. The distributed 
NLMS algorithm has a complexity 0{NMI), while the 
complexity of the sparse diffusion NLMS algorithm 0 is 
0{3NMI). For the distributed compressive sensing algorithm 
of ll24l . the computational complexity is 0{NMI + ND^I). 
In the proposed DCE scheme, only the local compressed 
estimator with D parameters will be transmitted through 
the network, which means the transmission requirement is 
greatly reduced as compared with the standard schemes that 

transmit with M parameters. 

TABLE I 

The Proposed DCE Scheme 

Initialize: d>fc{l)=0 

For each time instant 2=1,2, .... I-1 

For each node fc=l,2, ..., N 

i/ifc(i) = + fJ.ii)el{i)xk(i) 

where efc(i) = dfc(i) - (i)xk(i), 

dk(i) = u3gXk(i) + nfe(i) = + rafc(i) 

and is the D X M random measurement matrix 

end 

For each node fc=l,2, ..., N 
l^Nk 

end 

end 

After the final iteration I 

For each node fc=l,2, ..., N 

where is the final decompressed estimator. 

end 


IV. Measurement Matrix Optimization 
To further improve the performance of the proposed DCE 
scheme, an optimization algorithm for the design of the 
measurement matrix which is now time-variant, is 

developed here. Unlike prior work ll2^ . 1^ . this optimization 
is distributed and adaptive. Let us consider the cost function 

J = E{|efe(z)|2} = E{|dfc(*)-yfc(*)|2} 

= E{|4(*)|"} - E{dUt)ykm - E{dkit)yim dO) 

where t/fe(i) = {i)xk{i). To minimize the cost function, 

we need to compute the gradient of J with respect to 


and equate it to a null vector, i.e., V = 0. As a result, 

only the first three terms in (fTOl i need to be considered. Taking 
the first three terms of (fTOl) we arrive at 

E{|4d)p} - E{dU^)yki^)} - E{dki^)yU^)} 

= E{\uJ^^^{^)xki^) + nki^m 

- ii)xkii) + nkii))*yk{i)} 

- ^{{^0 ^k {i)xk{^ + nk{i))yl{i)} 

+ ^{{^0 ^k i^)xk{i))n*k{i)} + E{\nk{i)\^} 

- ^k ii)^kii)yykii)} - E{nl{i)ykii)} 

- ^{{^o^k y)xk{i))y*k{i)} - E{nk{i)yl{i)}. ( 11 ) 

Because the random variable nk{i) is statistically independent 
from the other parameters and has zero mean, dnii can be 
further simplified as 

E{|4(*)in - E{diy)ykm - E{4(*)2/d*)} 

= (*)S,(*)n + alk - E{(a;?$f 

-E{{u;y<i>y{{)xky))y*k{i)}. ( 12 ) 

Then, we have 

= Rk{i)^k(i)Ri^o ~ Pk(i), ( 13 ) 

where Rk{i) = E{xk{i)xy[i)}, R^^^ = Ejwou;^} and 
Pk{i) = E{ 2 /^(z)Sfc(i)u;^}. Equating (fOl l to a null vector, 
we obtain 

Rk{i)^k{i)RLJo ~ Pk{i) = ( 14 ) 

= Rl\yPk{i)RZl- ( 15 ) 

The expression in dull cannot be solved in closed-form 
because oiq is an unknown parameter. As a result, we employ 
the previous estimate uikii) to replace ljq. However, 
and depend on each other, thus, it is necessary to iterate 

(flSl l with an initial guess to obtain a solution. In particular, we 
replace the expected values with instantaneous values. Starting 
from (fTSl l. we use instantaneous estimates to compute 

Rk{i) = Xk{i)xy{i), ( 16 ) 

R^^ = LjQLjy ( 17 ) 

and 

Pk{i) = yl{i)xkii)i^o ■ ( 18 ) 

According to the method of steepest descent ll34l . the updated 
parameters of the measurement matrix at time i + 1 are 

computed by using the simple recursive relation 

+ 1 ) = 

= + r][Pk{i) - Rk{i)^k{i)RkJo] ( 19 ) 

= + r][yl{i)xk{i)ujy - Xk{i)xy {i)^k{i)i^oi^o]- 

where y is the step size and u>q is the M x 1 unknown 
parameter vector that must be estimated by the network. Then, 
the parameter vector u>k{i) is used to reconstruct the estimate 
of ljq as follows 


= /0Mp{‘^fc(*)}, 


( 20 ) 
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where the operator /omp{-} denotes the OMP reconstruction 
algorithm. Note that other reconstruction algorithms could 
also be employed. Replacing ujq by we arrive at the 

expression for updating the measurement matrix described by 

+ 1 ) = 

- Xk[i)x^ (O] • (21) 

The computational complexity of the proposed scheme with 
measurement matrix optimization is 0{NDI + ND^I). 

V. Simulations 

We assess the proposed DCE scheme and the measurement 
matrix optimization algorithm in a WSN application, where a 
partially connected network with N = 20 nodes is considered. 
We compare the proposed DCE scheme with uncompressed 
schemes, including the distributed NLMS (dNLMS) algorithm 
(normalized version of El), sparse diffusion NLMS algo¬ 
rithm 0, sparsity-promoting adaptive algorithm IS), and the 
distributed compressive sensing algorithm ll24l . in terms of 
MSE performance. Note that other metrics such as mean- 
square deviation (MSD) could be used but result in the same 
performance hierarchy between the analyzed algorithms. 

The input signal is generated as Xk{i) = \xk{i) Xk{i — 
1) ... Xk{i - M + 1)]'^ and Xk{i) = Uk{i) + akXk{i- 

1), where ak is a correlation coefficient and Uk{i) is a white 
noise process with variance ^ = 1 — |ctfcP, to ensure the 
variance of Xk{i) is cr^ = 1. The compressed input signal is 
obtained by Xk{i) = ^kXk{i)- The measurement matrix ^k 
is an i.i.d. Gaussian random matrix that is kept constant. The 
noise samples are modeled as complex Gaussian noise with 
variance ^ = 0.001. The unknown M x 1 parameter vector 
ujq has sparsity S, where M=5Q, D=IQ and S=3. The step size 
po for the distributed NLMS, distributed compressive sensing, 
sparse diffusion LMS and the proposed DCE algorithms is 
0.45. The parameter that controls the shrinkage in 0 is set 
to 0.001. Eor 0, the number of hyperslabs equals 55 and the 
width of the hyperslabs is 0.01. 

Eig. [3] illustrates the comparison between the DCE scheme 
with other existing algorithms, without the measurement ma¬ 
trix optimization. It is clear that, when compared with the 
existing algorithms, the DCE scheme has a significantly faster 
convergence rate and a better MSE performance. These ad¬ 
vantages consist in two features; the compressed dimension 
brought by the proposed scheme and CS being implemented 
in the estimation layer. As a result, the number of parameters 
for transmission in the network is significantly reduced. 

In the second scenario, we employ the measurement matrix 
optimization algorithm to in the DCE scheme. The parameter 
p for the measurement matrix optimization algorithm is set 
to 0.08 and all other parameters remain the same as in 
the previous scenario. In Eig. H) we observe that with the 
help of the measurement matrix optimization algorithm, DCE 
can achieve a faster convergence when compared with DCE 
without the measurement matrix optimization. 

In the third scenario, we compare the DCE scheme with the 
distributed NLMS algorithm with different levels of resolution 
in bits per coefficient, reduced dimension D and sparsity level 
S. The x-axis stands for the reduced dimension D and their 
corresponding sparsity level S can be found in Eig.|5] In Eig.|5] 


it is clear that with the increase of the sparsity level S the MSE 
performance degrades. In addition, the MSE performance will 
increase when the transmission has more bits per coefficient. 
Eor the DCE scheme, the total number of bits required for 
transmission is D times the number of bits per coefficient, 
whereas for the distributed NLMS algorithm it is M times the 
number of bits per coefficient. A certain level of redundancy is 
required between the sparsity level and the reduced dimension 
due to the error introduced by the estimation procedure. 



Fig. 3. MSE performance against time 



Fig. 4. MSE performance against time with measurement matrix optimization 



Fig. 5. MSE performance against reduced dimension D for different levels 
of resolution in bits per coefficient 

VI. Conclusions 

We have proposed a novel DCE scheme and algorithms 
for sparse signals and systems based on CS techniques and 
a measurement matrix optimization. In the DCE scheme, the 
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estimation procedure is performed in a compressed dimension. 
The results for a WSN application show that the DCE scheme 
outperforms existing strategies in terms of convergence rate, 
reduced bandwidth and MSE performance. 
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